================================================================================
  Math Unikernel -- Capstone Project 2
  README.txt -- Code Directory Structure, Installation Guide, and User Manual
================================================================================

--------------------------------------------------------------------------------
TABLE OF CONTENTS
--------------------------------------------------------------------------------
  1. Project Overview
  2. Top-Level Directory Structure
  3. Detailed Component Descriptions
       3.1  math_unikernel_os/
       3.2  math_unikernel_vscode_extension/
       3.3  math_unikernel_benchmark_workload/
  4. Installation Guide
       4.1  Prerequisites
       4.2  Building the Unikernel OS
       4.3  Installing the VSCode Extension
       4.4  Building and Running the Benchmark Harness
  5. User Manual
       5.1  Running the Unikernel in QEMU
       5.2  Developing a Workload with the VSCode Extension
       5.3  Streaming a Workload to the Unikernel
       5.4  Running the Comparison Benchmarks
       5.5  Analyzing Benchmark Results

================================================================================
1. PROJECT OVERVIEW
================================================================================

Math Unikernel is a bare-metal x86-64 operating system kernel engineered
exclusively for high-performance numerical computation. Unlike general-purpose
operating systems, the unikernel exposes only the functionality required to
receive, execute, and report the results of math workloads: memory management
(physical and virtual), SIMD-accelerated math primitives, a lightweight
Ethernet driver, a framebuffer display, and a serial console.

Workloads are small, position-independent C programs compiled against a thin
"Kernel API" SDK.  They are transferred to the running unikernel over a raw
Ethernet link (custom EtherType 0x88B5), executed in place, and then the
unikernel returns to a polling state ready for the next workload.

The repository contains three independently buildable components:
  - math_unikernel_os                 The unikernel kernel itself
  - math_unikernel_vscode_extension   VSCode extension for workload development
  - math_unikernel_benchmark_workload Benchmark workloads + comparison harness

================================================================================
2. TOP-LEVEL DIRECTORY STRUCTURE
================================================================================

capstone_2_final_submission/
│
├── README.txt                         ← This file
│
├── math_unikernel_os/                 ← Kernel source and build system
│   ├── src/                           ← All C and Assembly kernel source files
│   │   ├── headers/                   ← Public header files for each subsystem
│   │   ├── nic_drivers/               ← Network interface card drivers
│   │   ├── kernel.c                   ← Kernel entry point (kernel_main)
│   │   ├── boot.S                     ← Assembly boot stub
│   │   ├── gdt.c / gdt_load.S         ← Global Descriptor Table
│   │   ├── idt.c / isr.S              ← Interrupt Descriptor Table + ISRs
│   │   ├── pmm.c                      ← Physical Memory Manager
│   │   ├── vmm.c                      ← Virtual Memory Manager (huge pages)
│   │   ├── loader.c                   ← Network workload loader (polling)
│   │   ├── network.c                  ← Ethernet frame reception
│   │   ├── pci.c                      ← PCI bus scanner / NIC detection
│   │   ├── pic.c                      ← Programmable Interrupt Controller
│   │   ├── display.c                  ← Framebuffer text renderer
│   │   ├── mathlib.c                  ← AVX/FMA math primitives
│   │   ├── simd.S                     ← SIMD enable/CPUID helpers
│   │   ├── lib.c                      ← Minimal libc helpers (memset, etc.)
│   │   └── io.c                       ← Port I/O and serial driver
│   ├── iso_root/                      ← Limine bootloader config + boot files
│   │   └── limine.cfg                 ← Bootloader configuration
│   ├── limine/                        ← Limine bootloader binaries (v7.x)
│   ├── Makefile                       ← Compiles all sources → kernel.elf
│   ├── linker.ld                      ← Linker script (higher-half kernel)
│   ├── build-iso.sh                   ← Wraps xorriso to produce bootable ISO
│   ├── clean.sh                       ← Removes all build artifacts
│   ├── qemu.sh                        ← Launches QEMU with correct flags
│   ├── run_all.sh                     ← make + build-iso.sh + qemu.sh in one step
│   ├── flash_drive.sh                 ← Writes ISO to a physical USB drive
│   └── README.md                      ← Quick-reference shell script guide
│
├── math_unikernel_vscode_extension/   ← Developer tooling (VSCode extension)
│   ├── src/                           ← TypeScript extension source
│   │   ├── extension.ts               ← Extension activation + command registration
│   │   ├── handlers.ts                ← Compile and Stream command logic
│   │   └── sidebarProvider.ts         ← Sidebar webview (MU Controls panel)
│   ├── out/                           ← Compiled JavaScript (deployable)
│   │   ├── extension.js               ← Transpiled extension entry point
│   │   ├── handlers.js                ← Transpiled handlers
│   │   └── sidebarProvider.js         ← Transpiled sidebar provider
│   ├── sdk/                           ← Workload SDK (copy into workload project)
│   │   ├── kernel_api.h               ← Kernel API struct definition
│   │   ├── linker_script.ld           ← Linker script for workload binaries
│   │   └── Makefile                   ← Makefile template for workloads
│   ├── python/
│   │   └── send.py                    ← Scapy script that streams workload over Ethernet
│   ├── .vscode/
│   │   └── launch.json                ← Extension development/debug config
│   ├── package.json                   ← Extension manifest + npm scripts
│   ├── tsconfig.json                  ← TypeScript compiler configuration
│   ├── eslint.config.mjs              ← ESLint configuration
│   └── README.md                      ← Extension feature summary
│
└── math_unikernel_benchmark_workload/ ← Benchmark workloads and results
    ├── workloads/                     ← Unikernel workload source files
    │   ├── bench_dot.c                ← Dot product benchmark (16M floats, 100 iters)
    │   └── bench_gemm.c               ← GEMM benchmark (2048x2048 matrix)
    ├── windows_linux_benchmark/       ← Baseline harness for comparison OSes
    │   ├── bench_harness.c            ← Cross-platform C benchmark (Linux + Windows)
    │   ├── bench                      ← Compiled Linux binary of the harness
    │   └── results.csv                ← Raw CSV results from Linux baseline run
    ├── sdk/                           ← Copy of workload SDK (same as extension/sdk)
    │   ├── kernel_api.h
    │   ├── linker_script.ld
    │   └── Makefile
    ├── plots.ipynb                    ← Jupyter notebook for result visualization
    ├── benchmark.png                  ← Bar chart: unikernel vs Linux/Windows
    ├── benchmark_comparison.png       ← Side-by-side comparison chart
    └── benchmark_full.csv             ← Consolidated benchmark results

================================================================================
3. DETAILED COMPONENT DESCRIPTIONS
================================================================================

--- 3.1  math_unikernel_os/ ---

PURPOSE
  The complete source and build system for the bare-metal unikernel.  The kernel
  is compiled with x86_64-elf-gcc using a freestanding, no-stdlib configuration.
  It boots via the Limine bootloader (v7.x), initializes hardware, and then
  enters an infinite poll → execute → poll loop.

SOURCE FILES  (math_unikernel_os/src/)
  kernel.c          Main kernel entry; hardware init sequence; state machine.
  boot.S            Assembly entry point called by Limine; sets up stack.
  gdt.c / gdt_load.S  64-bit GDT setup.
  idt.c / isr.S     64-bit IDT; interrupt service routines.
  pmm.c             Bitmap-based physical frame allocator.
  vmm.c             4-level page table manager; huge-page allocator.
  loader.c          Receives workload size + payload over Ethernet frames.
  network.c         Raw Ethernet frame reception; interrupt-driven DMA.
  pci.c             PCI bus scan; locates NIC by class/vendor ID.
  pic.c             PIC remapping (IRQs 0-15 → vectors 32-47).
  display.c         Pixel-perfect 8x8 font framebuffer text output.
  mathlib.c         AVX/FMA dot product, GEMM, SpMV (SSE+AVX enabled for
                    this file only; rest of kernel has SSE disabled).
  simd.S            Enables x87/SSE/AVX state in CR0/CR4/XCR0.
  lib.c             memset, memcpy, itoa, strlen equivalents.
  io.c              inb/outb; 8250-compatible serial UART.
  nic_drivers/      Per-NIC driver implementations:
    rtl8139.c/.h    RTL8139 driver (QEMU default NIC).
    i219.c/.h       Intel I219 driver (physical hardware).

DEPLOYABLE ARTIFACTS
  kernel.elf        ELF binary produced by 'make'.
  *.iso             Bootable ISO produced by build-iso.sh.
  (ISO can be flashed to USB with flash_drive.sh or booted in QEMU.)

BUILD SCRIPTS
  Makefile          Compiles all .c/.S → .o → kernel.elf (x86_64-elf-gcc).
  build-iso.sh      Calls xorriso + limine-deploy to wrap kernel.elf into ISO.
  clean.sh          Removes kernel.elf, all .o files, and .iso files.
  qemu.sh           Launches QEMU: headless, COM1→stdout, 512 MB RAM,
                    RTL8139 NIC on user network.
  run_all.sh        Convenience: make && ./build-iso.sh && ./qemu.sh.
  flash_drive.sh    Writes ISO to a physical block device (requires sudo).

--- 3.2  math_unikernel_vscode_extension/ ---

PURPOSE
  A VSCode extension that provides a sidebar panel ("MU Controls") and two
  commands — Compile and Stream — to streamline the workload development loop.

SOURCE FILES  (math_unikernel_vscode_extension/src/)
  extension.ts       Registers commands and the sidebar webview; activates on
                     VSCode startup.
  handlers.ts        'compile' invokes x86_64-elf-gcc + objcopy via a terminal.
                     'stream' calls python/send.py with the compiled binary, the
                     target MAC address, and the network interface from settings.
  sidebarProvider.ts Renders the MU Controls webview panel with Compile/Stream
                     buttons.

DEPLOYABLE FILES  (math_unikernel_vscode_extension/out/)
  extension.js       Transpiled entry point (loaded by VSCode at runtime).
  handlers.js        Transpiled handlers.
  sidebarProvider.js Transpiled sidebar provider.
  (The out/ directory is the deployable artifact; TypeScript sources in src/ are
  not executed directly.)

SDK  (math_unikernel_vscode_extension/sdk/)
  kernel_api.h       Defines KERNEL_API_ADDRESS (0x10000000), the kernel_api_t
                     struct with function pointers exposed by the kernel, VMM
                     flag constants, and the MAIN macro that places the workload
                     entry point in the .entry section.
  linker_script.ld   Places the .entry section at address 0x0 so objcopy
                     produces a flat binary with the entry point at offset 0.
  Makefile           Builds workload.c → workload.elf → workload.bin.

PYTHON SCRIPT  (math_unikernel_vscode_extension/python/send.py)
  Uses Scapy to send workload bytes to the unikernel over raw Ethernet.
  Protocol: custom EtherType 0x88B5; magic header 0x474F2121 ("GO!!") followed
  by an 8-byte little-endian payload length, then the binary in 1486-byte chunks.

CONFIGURATION  (VSCode Settings)
  mathUnikernel.targetMac    MAC address printed by the unikernel on boot.
  mathUnikernel.interface    Host network interface (e.g., eno1, eth0, en0).

--- 3.3  math_unikernel_benchmark_workload/ ---

PURPOSE
  Contains the unikernel workload source files used for benchmarking, a
  cross-platform baseline harness for Linux and Windows, and the collected
  benchmark results with analysis notebooks.

WORKLOAD SOURCE FILES  (math_unikernel_benchmark_workload/workloads/)
  bench_dot.c    Dot product on 16 million single-precision floats, 100
                 iterations (~3.2 billion FLOPs total).
  bench_gemm.c   General matrix multiply on 2048×2048 matrices, tiled for
                 cache efficiency, 10 iterations.
  Both files use the Kernel API (via sdk/kernel_api.h) and are compiled to
  flat position-independent binaries with the SDK Makefile.

COMPARISON HARNESS  (math_unikernel_benchmark_workload/windows_linux_benchmark/)
  bench_harness.c  Implements identical dot product, GEMM, and SpMV workloads
                   using the same AVX/FMA intrinsics as the kernel's mathlib.c.
                   Portable to Linux (gcc) and Windows (MSVC cl).  Outputs CSV
                   to stdout and human-readable statistics to stderr.
  bench            Pre-compiled Linux x86-64 binary of bench_harness.c.
  results.csv      Raw CSV results from a Linux baseline run.

SDK  (math_unikernel_benchmark_workload/sdk/)
  Identical copy of the workload SDK (see Section 3.2).

RESULTS AND ANALYSIS
  benchmark_full.csv         Consolidated CSV: unikernel + Linux + Windows runs.
  benchmark.png              Bar chart comparing mean execution time per benchmark
                             across the three environments.
  benchmark_comparison.png   Side-by-side chart highlighting unikernel advantages.
  plots.ipynb                Jupyter notebook that reads benchmark_full.csv and
                             generates the two PNG charts.

================================================================================
4. INSTALLATION GUIDE
================================================================================

--- 4.1  Prerequisites ---

The following tools must be installed on the HOST (development) machine.

  a) Cross-compiler toolchain
       x86_64-elf-gcc   (GCC targeting bare-metal x86-64; no host libc)
       x86_64-elf-ld    (comes with the cross-GCC package)

     On Arch Linux:
       pacman -S x86_64-elf-gcc

     On Ubuntu/Debian, build from source or use a pre-built cross toolchain:
       See: https://github.com/lordmilko/i686-elf-tools (adapt for x86_64-elf)

  b) ISO creation tools
       xorriso          (for building the bootable ISO)

     On Arch:   pacman -S xorriso
     On Ubuntu: apt install xorriso

  c) QEMU (for emulated testing)
       qemu-system-x86_64

     On Arch:   pacman -S qemu-system-x86
     On Ubuntu: apt install qemu-system-x86

  d) Python 3 + Scapy (for workload streaming)
       pip install scapy
     Root or CAP_NET_RAW is required to send raw Ethernet frames.

  e) VSCode (for the extension)
       Version 1.115.0 or later.
       Download from: https://code.visualstudio.com/

  f) Node.js + npm (to build/modify the extension from TypeScript source)
       Node.js 18 LTS or later recommended.

  g) Jupyter + matplotlib (optional, for re-generating benchmark plots)
       pip install jupyter matplotlib pandas

--- 4.2  Building the Unikernel OS ---

Step 1 — Open a terminal and navigate to the OS directory:
  cd capstone_2_final_submission/math_unikernel_os

Step 2 — Compile the kernel:
  make

  Expected output:
    x86_64-elf-gcc ... -c src/boot.S -o src/boot.o
    x86_64-elf-gcc ... -c src/kernel.c -o src/kernel.o
    ... (one line per source file)
    x86_64-elf-ld -T linker.ld ... -o kernel.elf

  If you see "command not found: x86_64-elf-gcc", the cross-compiler is not
  installed or not on your PATH.  Re-check Step 4.1a.

Step 3 — Build the bootable ISO:
  ./build-iso.sh

  This script:
    - Copies kernel.elf into iso_root/
    - Calls xorriso to package the ISO with the Limine bootloader files
    - Runs limine-deploy to install the bootloader into the ISO's MBR

  Expected output ends with: "Build complete: math_unikernel.iso"

Step 4 — (Optional) Run in QEMU to verify the build:
  ./qemu.sh

  The kernel will print its initialization log to your terminal (serial output
  redirected to stdout by QEMU).  You should see lines such as:
    [OK] Limine handshake
    [OK] GDT initialized
    [OK] IDT initialized
    [OK] PMM initialized
    [OK] VMM initialized
    [OK] Display initialized
    [OK] Serial driver initialized
    [OK] Network controller found
    [OK] Timer calibrated
    [POLLING] Waiting for workload...

  The kernel is now waiting for a workload binary over Ethernet.
  Press Ctrl+C to stop QEMU.

Step 5 — (Optional) Flash to a physical USB drive:
  CAUTION: This will overwrite the device.  Double-check the device path.
  sudo ./flash_drive.sh /dev/sdX   (replace /dev/sdX with your drive)

--- 4.3  Installing the VSCode Extension ---

OPTION A — Install from the pre-built out/ directory (no build step needed)

  The out/ directory contains the compiled JavaScript that VSCode loads.  To
  install the extension into VSCode as a local, unpacked extension:

  Step 1 — Copy the entire math_unikernel_vscode_extension/ folder to the
           VSCode extensions directory:

    Linux / macOS:
      cp -r math_unikernel_vscode_extension ~/.vscode/extensions/mu-0.0.1

    Windows (PowerShell):
      Copy-Item -Recurse math_unikernel_vscode_extension `
        "$env:USERPROFILE\.vscode\extensions\mu-0.0.1"

  Step 2 — Restart VSCode.

  Step 3 — Verify installation: open the Extensions panel (Ctrl+Shift+X) and
           search for "math-unikernel-VSCode". It should appear as installed.

  Step 4 — Configure streaming parameters via File → Preferences → Settings
           (Ctrl+,) and search for "Math Unikernel":
             mathUnikernel.targetMac   Set to the MAC shown by the kernel on boot
             mathUnikernel.interface   Set to your host NIC (e.g., eno1, eth0)

OPTION B — Build from TypeScript source (for development/modification)

  Step 1 — Navigate to the extension directory:
    cd capstone_2_final_submission/math_unikernel_vscode_extension

  Step 2 — Install npm dependencies:
    npm install

  Step 3 — Compile TypeScript to JavaScript:
    npm run compile

    This runs tsc and writes output to out/.

  Step 4 — Follow OPTION A Steps 1-4 to install the compiled extension.

  Step 5 — To develop with hot-reload, open the folder in VSCode and press F5
           to launch the Extension Development Host (uses .vscode/launch.json).

--- 4.4  Building and Running the Benchmark Harness ---

The comparison benchmark harness (bench_harness.c) compiles on Linux and
Windows with the same SIMD intrinsics used by the unikernel's math library.

  ON LINUX:
    cd capstone_2_final_submission/math_unikernel_benchmark_workload/windows_linux_benchmark
    gcc -O3 -mavx -mfma -o bench bench_harness.c -lm

    A pre-compiled binary named 'bench' is already included.

  ON WINDOWS (Developer Command Prompt):
    cl /O2 /arch:AVX2 bench_harness.c

  To build unikernel workloads (bench_dot.c, bench_gemm.c):
    cd capstone_2_final_submission/math_unikernel_benchmark_workload
    cp workloads/bench_dot.c workload.c
    make
    # Produces workload.bin — ready to stream to the unikernel

================================================================================
5. USER MANUAL
================================================================================

--- 5.1  Running the Unikernel in QEMU ---

  From math_unikernel_os/:
    ./qemu.sh

  The QEMU instance is configured as follows:
    - No display window (headless mode; display output also goes to serial)
    - COM1 serial port is redirected to host stdout
    - 512 MB RAM allocated
    - One RTL8139 NIC on the QEMU user network (NAT)

  The kernel prints colored status messages and then enters the POLLING state.
  It will stay in this state until a valid workload is received.

  To terminate: Ctrl+C in the terminal running qemu.sh.

--- 5.2  Developing a Workload with the VSCode Extension ---

  Step 1 — Open your workload project folder in VSCode.
           The folder must contain workload.c and an sdk/ subdirectory
           with kernel_api.h, linker_script.ld, and Makefile.
           Copy the sdk/ folder from math_unikernel_vscode_extension/sdk/
           or math_unikernel_benchmark_workload/sdk/.

  Step 2 — Open the Explorer sidebar.  You will see the "MU Controls" panel
           at the bottom with two buttons: Compile and Stream.

  Step 3 — Write your workload in workload.c.  The entry point must be
           declared with the MAIN macro:

             #include "sdk/kernel_api.h"
             MAIN int main(void) {
                 kernel_api_t* api = (kernel_api_t*)KERNEL_API_ADDRESS;
                 api->prints("Hello from workload!\n");
                 return 0;
             }

           The MAIN macro places main() in the .entry section at address 0x0
           of the flat binary so the kernel can jump directly to it.

  Step 4 — Click "Compile" (or run Command Palette → "MU: Compile").
           This runs the SDK Makefile:
             make → workload.elf → workload.bin (via objcopy)

  Step 5 — With the unikernel running (in QEMU or on hardware), click "Stream"
           (or "MU: Stream").  The extension calls python/send.py which sends
           workload.bin to the configured MAC address over the configured
           network interface.

  Step 6 — Observe the unikernel output in the QEMU terminal or serial console.

--- 5.3  Streaming a Workload to the Unikernel ---

  The streaming protocol works as follows:
    1. A "handshake" frame is sent with magic bytes 0x474F2121 ("GO!!").
    2. An 8-byte little-endian integer specifies the payload byte count.
    3. The binary is chunked into 1486-byte payloads and sent as raw Ethernet
       frames with EtherType 0x88B5.

  The kernel's loader.c polls each received frame and writes it to a huge-page
  allocation in virtual memory.  When all bytes are received, it jumps to the
  start of the allocation and executes the workload.

  To stream manually (without the VSCode extension):
    sudo python3 math_unikernel_vscode_extension/python/send.py \
      workload.bin <target_mac> <interface>

  Example:
    sudo python3 send.py workload.bin de:ad:be:ef:00:01 eno1

  Note: root privileges (or CAP_NET_RAW) are required for Scapy to send raw
  Ethernet frames.

--- 5.4  Running the Comparison Benchmarks ---

  LINUX BASELINE:
    cd math_unikernel_benchmark_workload/windows_linux_benchmark
    ./bench > results.csv

    The binary is already compiled.  Results are written as CSV to stdout and a
    human-readable summary to stderr.  Redirect to capture both:
      ./bench > results.csv 2> summary.txt

  WINDOWS BASELINE:
    After compiling bench_harness.c with MSVC (see 4.4):
      bench.exe > results.csv

  UNIKERNEL BENCHMARKS:
    Each benchmark workload (bench_dot.c, bench_gemm.c) is streamed to the
    unikernel while it is running.  The unikernel prints timing results
    (cycles and milliseconds) to the serial console / display.

--- 5.5  Analyzing Benchmark Results ---

  The Jupyter notebook plots.ipynb reads benchmark_full.csv and generates the
  comparison charts.

  To re-run the notebook:
    cd math_unikernel_benchmark_workload
    jupyter notebook plots.ipynb

  Pre-generated charts are already included:
    benchmark.png             Per-benchmark mean time bar chart.
    benchmark_comparison.png  Side-by-side OS comparison chart.

  The CSV format for benchmark_full.csv is:
    benchmark, mean_ms, stddev_ms, min_ms, max_ms, p50_ms, p99_ms,
    mean_cycles, min_cycles, max_cycles

================================================================================
END OF README.txt
================================================================================
